Journal of Adult Education 




MOUNTAIN PLAINS 
ADULT I-DUC'ATION 


Arizona 

Colorado 

Idaho 

Montana 

Nevada 

New Mexico 

Utah 

Wyoming 


Volume 43, Number 1 
2014 


£5 


MOUNTAIN PLAINS 
ADULT EDUCATION 

ASSOCIATION 


Anzona 

Colorado 

Idaho 

Montana 

Nevada 

New Mexico 

Utah 

Wyoming 


Development of an Instrument to Measure Teaching Style in Japan: 
The Teaching Style Assessment Scale 

Fumiko Yoshida, Gary J. Conti, Toyoaki Yamauchi, and Takaaki Iwasaki 

Abstract 

Teaching style has been a popular concept for many years. Teaching style refers to the distinct 
qualities displayed by a teacher that are persistent from situation to situation regardless of the 
content. The Principles of Adult Learning Scale (PALS) has been used extensively in the West for 
measuring teaching style in relationship to the adult education literature, but no version has been 
available for use in Japan. Therefore, this research used PALS as the foundation for creating a 
new instrument to measure teaching style in Japan. Following a rigorous translation process, data 
were collected from a national sample of 1,111 nursing educators. Factor analysis and correlation 
were used to establish the construct and content validity of the new instrument. Reliability was 
established with Cronbach’s alpha. The new 30-item instrument was named the Teaching Style 
Assessment Scale and is available in both Japanese and English for use either for personal self- 
assessment or for research purposes. 


Background 

A crucial issue of Japanese nursing education is 
practicing Evidence-Based Nursing (Ministry of 
Health, Labor and Welfare, 2008). The amount of 
research on Evidence-Based Nursing has been 
increasing since the need for Evidence-Based Nursing 
was reported (Asakawa, 2011). The nurse practicing 
Evidence-Based Nursing acquires learning skills such 
as critical thinking and self-directed learning. These 
skills are essential for nursing competencies and 
require a learning environment in which teachers 
encourage learners to apply the higher-order cognitive 
skills in Bloom’s taxonomy of cognitive behaviors 
(Heimlich & Norland, 1994). 

Critical thinking is essential for Evidence-Based 
Nursing practice, and developing these skills requires 


a student-centered approach (Chipas, 1995; Schaeffer 
& Zygmount, 2003). The National Leagues for 
Nursing in the USA proposed the necessity for 
student-centered approaches to learning. This position 
is congruent with the long history of adult learning 
theory in the United States in which the learner- 
centered concepts of andragogy and self-directed 
learning form the twin pillars (Merriam, 2001). 

In his original description of andragogy, Knowles 
(1970) pointed out that “the behavior of the teacher 
probably influences the character of the learning 
climate more than any other single factor” (p. 41). 
Moreover, research has shown that “teaching does 
make a difference... .Teaching is the human 
connection between the content and the environment 
and the learners” (Heimlich & Norland, 1994, p. 109). 
There are two basic approaches for the teacher 
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making this human connection with the learners. 
These are the teacher-centered approach and the 
learner-centered approach (Conti, 2004). With the 
teacher-centered approach, “the teacher’s role is to 
design an environment which stimulates the desired 
behavior and discourages those that have been 
determined to be undesirable” (p. 77). While the 
teacher-centered approach focuses upon the actions of 
the teacher in planning and controlling the learning 
environment, the learner-centered approach is 
concerned with the personal development of each 
individual learner, and the focus therefore is upon the 
individual learner. “Although a teacher-centered 
approach is widely practiced in adult education, the 
learner-centered approach is strongly supported in the 
field’s literature” (p. 78). 

“Teaching style refers to the distinct qualities 
displayed by a teacher that are persistent from 
situation to situation regardless of the content” (Conti, 
2004, pp. 76-77). Teaching style is much broader than 
the specific teaching strategies and methods that are 
employed to accomplish a specific instructional 
objective (Conti, 1989, 2004). Teachers enter the 
teaching-learning transaction with a definite set of 
values (Brookfield, 1986), and these in turn influence 
the teacher’s beliefs about the nature of the learner, 
the purpose of the curriculum, and the role of the 
teacher in the classroom (Darkenwald & Merriam, 
1982). An awareness of one’s teaching style is 
important in order for teachers “to examine their 
beliefs about teaching and current teaching behavior 
in depth” (Heimlich & Norland, 1994, p. xi) so they 
can reflect critically upon their professional practice 
(Conti, 2004; Heimlich & Norland, 1994). Thus, “a 
knowledge of teaching style can make a difference in 
how teachers organize their classroom, how they deal 
with learners, and how well their students do in 
learning” (Conti, 1989, p. 3). 

The Principles of Adult Learning Scale (PALS) 
has been used extensively in the field of adult 
education for measuring teaching style (for example, 
see Brookfield, 1986; Heimlich & Norland, 1994; 
Merriam & Bierema, 2013). Since it was developed in 
1978, PALS has been used in over 100 research 
studies and in countless training activities. PALS is a 
44-item, summated rating scale that “measures the 


frequency with which one practices teaching/learning 
principles that are described in the adult education 
literature” (Conti, 2004, p. 79). 

Although PALS has been available for measuring 
teaching style in English-speaking countries, there has 
been no scale with which to measure a teacher’s 
teaching style in Japan. Therefore, it was necessary to 
develop a Japanese version of a teaching style 
instrument. Because PALS is based on the long- 
established theory base for adult learning theory, 
PALS was used as the foundation for this new 
instrument. The purpose of this research was to 
develop the Teaching Style Assessment Scale for 
teachers of nursing in Japan that could be used to 
identify their teaching style as either student-centered 
or teacher-centered. 

Research Methods 

The development of an instrument involves 
determining items for the instrument and then 
collecting validity and reliability data for these items 
(Gay, Mills, & Airasian, 2009). The Principles of 
Adult Learning Scale (PALS) was used as the source 
of items for the Teaching Style Assessment Scale 
(TSAS). The validity and reliability of these items 
was inferred from the extensive history and research 
record associated with PALS and from its use in 
numerous diverse setting. However, because of the 
cultural diversity between Japan and the Western 
countries in which PALS was developed and used, it 
was necessary to gather data on these items with a 
sample in Japan. 

The first task for making the TSAS was to 
translate the items in PALS from English to Japanese. 
This translation was accomplished in three steps. Step 
1 was to translate the original English items in PALS 
to Japanese. Step 2 was to back translate this Japanese 
translation to English in order to check its accuracy. 
Step 3 was to combine the individual translations into 
one scale. All steps were conducted by experts in 
Japanese, English, and nursing education. To further 
test the content validity of the translated items, they 
were examined by three English experts who are 
native Japanese from three universities. Seven 
professors, who were not in the nursing field, also 


12 



examined the translated version of the instrument for 
clarity. The final translated version was then pilot 
tested with 10 Japanese nursing educators from a 
university (7), a junior college (1), and a nursing 
diploma school (2) and with 7 teachers from outside 
the field of nursing. 

Comparing TSAS to PALS 

Data were gathered from a national Japanese 
sample to compare TSAS to the norms for PALS. To 
obtain a national sample of nursing educators, 2,256 
questionnaires were sent to nursing teachers at 363 
facilities. From these, 1,111 questionnaires were 
returned for a 49.2% response rate. Out of 1,111 
responses, 679 (61.1%) were from nursing diploma 
schools, 386 (34.8%) were from the university, and 
46 (4.1%) were from junior colleges. 

The scores on TSAS were significantly different 
from the norms for PALS (t = 84.4, df = 1,110, p < 
.001). The mean for TSAS was 114.3 with a standard 
deviation of 12.5, a median of 114, a mode of 108, 
and range of 68 to 162. The total score on PALS can 
be broken down into seven factor scores. The mean 
scores for the national sample for every factor were 
significantly lower on the TSAS than the norms for 
the factors on PALS. Thus, the scores for this sample 
were very different from the norms for PALS. 

Factor analysis was used to investigate the 
theoretical constructs, or factors, that might underlie 
the structure of the sample and to determine if it had 
the same underlying factor structure as PALS. For 
this analysis, the 44 items from the 1,111 responses 
on the TSAS were factor analyzed using a principal 
components analysis with a varimax rotation. Because 
the results were to be compared to PALS, the number 
of factors for the analysis was set at seven. In the 
analysis, all 44 items loaded into 7 factors that 
explained 44.3% of the variance in the analysis. 
However, the items in each factor were not the same 
as those in PALS. Many of the items continued to be 
correlated with each other in TSAS but in smaller 
clusters, and these clusters then joined other clusters 
in forming factors. Consequently, while the factors 
for TSAS and PALS are similar in the concepts that 
they represent, they are somewhat different in the 


items that make up these concepts. 

Cronbach’s alpha was calculated to determine the 
internal consistency reliability for the 44 items from 
the 1,111 responses on TSAS. This procedure 
produced an alpha of .68 and a standardized item 
alpha of .70. 

Final Form of TSAS 

The comparison of TSAS to norms for PALS 
indicated that TSAS was similar to PALS in that it 
was measuring the construct of teaching style in 
relationship to the adult education literature base but 
that TSAS was different from PALS. Therefore, the 
standard instrument construction procedures for 
establishing validity and reliability were followed for 
creating the final form of TSAS. This process 
recognized that “there are different types of evidence 
of validity” (Wiersma & Jurs, 2005, p. 327) and that 
“there are multiple ways to establish the various 
forms of test validity” (Gay & Airasian, 2000, p. 

169). 

Criterion-Related Validity 

Criterion-related validity is determined by 
comparing a test to a second test or other measure 
(Gay, Mills, & Airasian, 2009). The steps in creating 
TSAS from PALS and the analysis of the results of 
TSAS with a national sample of 1,111 established the 
concurrent form of criterion-related validity for TSAS 
by demonstrating that its wording is similar to PALS, 
both instruments are measuring similar factors, and 
TSAS is minimally reliable in doing this. Thus, the 
criterion-related validity of TSAS rests in its 
construction from PALS and in its comparison to 
PALS. 

Content Validity 

“Content validity is the degree to which a test 
measures an intended content area” (Gay, Mills, & 
Airasian, 2009, p. 155). It is concerned with both how 
relevant the items are to the content area and how 
well the items sample the overall content area. 
Content validity is often established by the judgment 
of experts, but statistical procedures also can be used. 

The items in TSAS were translated from PALS; 
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therefore, the content validity of PALS was inferred 
for TSAS. The content validity of PALS was 
established by the testimonies of a local jury and a 
national jury in the United States. The local jury was 
made up of three well-established professors of adult 
education, and the national jury "consisted of 10 
professors with a high degree of visibility in the field 
of adult education, with geographic dispersion 
throughout the country, and with philosophical 
heterogeneity" (Conti, 1982, pp. 139-140). These jury 
members not only made many contributions to the 
adult education literature base throughout their 
careers but also served in leadership roles in national 
organizations and on professional journals. These jury 
members included distinguished adult educators such 
as Malcolm Knowles, Alan Knox, and Robert Smith. 
In the judgment of these distinguished adult education 
scholars, the items in PALS reflect the “adult 
education learning principles that are congruent with 
the collaborative teaching-learning mode....in which 
authority for curriculum formation is shared by the 
learner and the practitioner” (pp. 135-136). 

Statistical analysis was used to measure the 
degree to which each item is related to the 
measurement of the intended content and to which the 
items sample the overall content being measured 
(Gay, Mills, & Airasian, 2009). As with PALS, the 
items in TSAS are summed to produce a total score 
that represents the degree to which adult education 
practitioners accept and adhere to learner-centered 
principles in the adult education literature (Conti, 
1982). Content validity addresses how well each of 
the items in TSAS relates to this total concept of 
teaching style. In order to establish this content 
validity, the items were analyzed by examining the 
correlation between individual item responses and the 
total score on TSAS. This procedure was used in the 
original development of PALS, and this procedure is 
appropriate “because each item is part of the overall 
concept, and in order for the item to be useful, it must 
contribute to the total score. In order to do this, it 
must have a moderate to strong positive correlation” 
(Nichols-Sharpe, 2004, pp. 127-128). 

Several correlations were calculated to examine 
the relationship between the response on each 
individual item for the 1,111 participants in the 


national sample and the individual’s total score on 
TSAS. A stepwise procedure was used that is similar 
to that used in regression analysis (Kachigan, 1991; 
Sheskin, 2007); however, this procedure was used for 
removing items from the original 44 items in TSAS. 
That is, after each correlation was calculated, the 
results were examined, and the item with the lowest 
correlation was removed if it did not have at least a 
positive .2 correlation with the total score. For the 
next step, a new total score was calculated without 
this item. After repeating this procedure 14 times, all 
items correlated at least at the .2 level with the total 
score. The following items were removed: 2, 4, 7, 11, 
12, 13, 21, 26, 27, 33, 37, 38, 40, and 41; all of these 
were negative items. 

Construct Validity 

Construct validity “reflects the degree to which a 
test measures an intended hypothetical construct” 
(Gay, Mills, & Airasian, 2009, p. 157); for TSAS, this 
is the construct of teaching style as it relates to 
support of the collaborative mode in the adult 
education literature (Conti, 1982). The construct 
validity was established in two ways. 

First, just as with content validity, the construct 
validity of the items in TSAS can be inferred from 
PALS. The local and national juries that testified to 
the content validity of PALS also testified to the 
construct validity of PALS (Conti, 1982). The 
positive judgement of these major theorists and of 
journal editors strongly supports the construct validity 
of the items from PALS that were used in the final 
version of TSAS. 

Second, factor analysis was used to identify the 
underlying elements composing teaching style as 
measured by TSAS. This factor analysis used the 
responses to the final 30-item form of TSAS. The 
responses from the 1,111 participants in the national 
sample were factor analyzed using a principal 
components analysis with a varimax rotation. Because 
the analysis produced a solution with 6 factors with 
eigenvalues greater than 1, separate analyses were run 
for 2 to 6 factors to explore for the best fit of the data 
with the final form of TSAS. The 5-factor solution 
was judged the best fit for the data because it 
produced the best distribution of items in the factors. 
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All 30 items loaded into 5 factors that explained 
45.3% of the variance in the analysis. The factor 
loadings ranged 


from .75 to .33 and were distributed as follows: .70 to 
.79-5, .60 to .69-9, .50 to .59-8, .40 to .49-7, and .30 
to .39-1 (see Table 1). 


Table 1: Items and Factors in the Teaching Style Assessment Scale 


Load 

Item 

Description 

Factor 1: Participation in the Learning Process 

0.75 

23 

gear my instructional objectives to match the individual abilities and needs of the students. 

0.75 

18 

help my students develop short-range as well as long-range objectives. 

0.73 

16 

have individual conferences to help students identify their educational needs. 

0.73 

17 

let each student work at his/her own rate regardless of the amount of time it takes him/her to learn a new concep 

0.67 

25 

allow a student's motives for participating in continuing education to be a major determinant in the planning < 
anting objectives. 

0.64 

26 

have my students identify their own problems that need to be solved. 

0.55 

1 

allow students to participate in developing the criteria for evaluating their performance in class. 

0.53 

9 

allow students to participate in making decisions about the topics that will be covered in class. 

0.42 

22 

plan activities that will encourage each student's growth from dependence on others to greater independence. 

Factor 2: Relating to Experience 

0.73 

30 

teach units about problems of everyday living. 

0.64 

24 

encourage my students to ask questions about the nature of their society. 

0.61 

27 

organize adult learning episodes according to the problems that my students encounter in everyday life. 

0.59 

29 

help students relate new learning to their prior experiences. 

0.43 

8 

plan learning episodes to take into account my students' prior experiences. 

Factor 3: Climate Building (Create Learning Climate) 

0.65 

5 

participate in the informal counseling of students. 

0.60 

7 

arrange the classroom so that it is easy for students to interact. 

0.52 

3 

help students diagnose the gaps between their goals and their present level of performance. 

0.48 

15 

accept errors as a natural part of the learning process. 

0.47 

12 

encourage dialogue among my students. 

0.45 

14 

utilize the many competencies that most adults already possess to achieve educational objectives. 

Factor 4: Learner-Centered Activities 

0.70 

6 

use lecturing as the best method for presenting my subject material to adult students. 

0.68 

4 

provide knowledge rather than serve as a resource person. 

0.55 

10 

use one basic teaching method because I have found that most adults have a similar style of learning. 

0.52 

20 

use methods that foster quiet, productive deskwork. 

0.51 

21 

use tests as my chief method of evaluating students. 

0.49 

13 

use written tests to assess the degree of academic growth in learning rather than to indicate new directions ft 
arning. 

Factor 5: Personalizing Instruction 

0.63 

2 

allow older students more time to complete assignments when they need it. 

0.58 

19 

allow my students to take periodic breaks during the class. 

0.40 

11 

use different techniques depending on the students being taught. 

0.33 

28 

use different materials with different students. 
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The five factors were named based upon their 
factor loadings. This process was supplemented by 
comparing the loadings in TSAS to the item 
distribution in the factors in PALS. The five factors in 
the final form of TSAS were named as follows: Factor 
1: Participation in the Learning Process; Factor 2: 
Relating to Experience; Factor 3: Climate Building (or 
Create Learning Climate); Factor 4: Learner-Centered 
Activities; and Factor 5: Personalizing Instruction. 
Because of the resemblance of the TSAS factors to the 
PALS factors, the TSAS factors were named the same 
as five of the PALS factors. 

Reliability 

Cronbach’s alpha was used to establish the 
reliability of the 30-item form of TSAS. It was 
calculated by using the 30 items from the 1,111 
responses on TSAS. This procedure produced an 
alpha of .86 and a standardized item alpha of .87. 
These coefficients indicate strong internal consistency 
reliability for the final form of TSAS and are similar 
to the high reliability coefficients found in research 
with PALS. 

Norms for TSAS 

As a result of the development and validation 
process, TSAS is a 30-item summated rating scale that 
measures the frequency with which one practices 
teaching/learning principles that are described in the 
adult education literature. In order to provide a 
reference for interpreting the total score on TSAS and 
its factors, the means and standard deviations were 
calculated for TSAS and its five factors (see Table 2). 
These descriptive statistics provide norm-referenced 
scoring based on the normal distribution against which 
a person’s performance on TSAS can be compared to 
that of the 1,111 participants in the national sample 
(Gay, Mills, & Airasian, 2009). 


Table 2: Mean and Standard Deviation for TSAS 
and Factors 


Statistic 

TSAS 

Factor 

1 

2 

3 

4 

5 

Mean 

81 

22 

17 

23 

10 

9 

Std. Dev. 

15 

7 

3 

3 

4 

3 


Scoring TSAS 

TSAS is a 30-item summated rating scale. 
Although PALS uses a 6-point Likert-type scale 
ranging from Always to Never (Conti, 2004, p. 79), 
the scale has been reversed for TSAS because most of 
the negative items from PALS have been removed 
through the validation process, and reversing the scale 
greatly reduces the number of items that need to be 
rescored. Therefore, the 6-point Likert-type scale for 
TSAS is as follows: 0-Never, 1-Almost Never, 2— 
Seldom, 3-Often, 4-Almost Always, and 5-Always. 

The first step in scoring TSAS is to rescore the 
negative items. Items number 4, 6, 10, 13, 20, and 21 
are negative items. For these negative items, the 
following values are assigned: Always=0, Almost 
Always=l, Often=2, Seldom=3, Almost Never=4, and 
Never=5. Omitted items are assigned a neutral value 
of 2.5; this value puts the response in the middle of 
the scale and does not skew the overall score toward 
either the teacher-centered or learner-centered side. 

After the negative items are rescored, the total 
score is obtained by summing the values of the 
responses to all 30 items. Scores above the mean of 81 
indicate a tendency toward the learner-centered mode 
while scores below 81 imply support of the teacher- 
centered approach. Factor scores are computed and 
interpreted in a similar fashion. 
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Discussion 

Teaching style has been a popular concept in the 
educational literature for many years. A primary 
reason for this is that teachers are professionals and 
care deeply about what they are doing in the teaching¬ 
learning transaction. Moreover, many teachers of 
adults have not had the formal training in education 
that brings them into contact with the field’s literature 
base. Valid and reliable instruments such as the PALS 
provide an objective tool for teachers to assess their 
classroom practices and their beliefs about these 
practices. 

For many years, PALS has been a proven 
instrument for measuring teaching style for the 
teachers of adults. However, a new valid and reliable 
instrument, TSAS, now exists. TSAS was derived 
from PALS and consequently carries with it the long 
history of validity and reliability of PALS. 

TSAS is similar to PALS but has some important 
differences. It is 31.8% shorter and contains 2 fewer 
factors. The items in TSAS combine in a slightly 
different fashion to form five similar factors to those 
in PALS. The poles of the scale have been reversed in 
TSAS, and because TSAS contains less negative items 
than PALS, it is easier to score. 

TSAS can be used for either self-assessment or as a 
research tool. TSAS not only provides an overall score 
that indicates how teachers relate to the concepts in the 
adult education literature base, but it also provides 
five factor scores that identify specific classroom 
behaviors that make up this style. By critically 
analyzing their responses to each item in each of the 
factors, teachers can reflect upon their classroom 
actions related to that style and upon consistency in 
their style. This can then be related to adult learning 
theory. 

TSAS can also be used in research. As with PALS, 
TSAS can be used in a variety of situations and 
settings that involve adult learners. In studies using 
PALS, “60.2% have been descriptive in nature while 
39.8% were relational studies” (Byrd, 2010, pp. 91- 
92). Descriptive studies are needed to supply a 
baseline for organizations and agencies to provide a 
better understanding of current practices before 
professional development activities are undertaken; 
past research indicates that this research can be very 


diverse. Relational studies examine the relationship of 
teaching style to other variables, and PALS relational 
studies have addressed variables such as beliefs, 
distance, and student outcomes. TSAS can be used in 
similar ways as well as in studies that examine the 
relationship of PALS and TSAS to each other in 
various settings and in studies that further confirm the 
norms for TSAS. 

Thus, TSAS is a new, valid, and reliable 
instrument that can be used for measuring teaching 
style practices both at the individual and 
organizational level. The stimulus for its development 
was the need to assess the teaching style of nursing 
educators in order to design professional development 
activities based on adult learning theory that foster 
critical thinking skills to implement Evidence-Based 
Nursing in Japan. However, the result has been the 
development of a statistically strong instrument that 
can be used in any situation involving the adult 
teaching-learning transaction. Adult education 
practitioners are encouraged to use TSAS in their 
daily practice and in their research. Such use can 
contribute to improved professional practice and to 
expanding the adult education knowledge base. 
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